Matthew Leeds [Fri, 28 Sep 2018 01:07:51 +0000 (18:07 -0700)]
lib/repo: Allow disabling lock timeout
Currently the locking code checks if the value -1 was set for the config
key "lock-timeout-secs" and if so, a thread trying to acquire a lock
will block indefinitely. Positive values specify how long to attempt to
acquire a lock in a non-blocking way (the attempt is made once every
second). But when the value is read from the config file,
g_ascii_strtoull() is used, which converts it to an unsigned integer.
This commit makes libostree use g_ascii_strtoll() instead, so that it's
possible to set that key to -1 as intended.
Matthew Leeds [Mon, 24 Sep 2018 20:57:42 +0000 (13:57 -0700)]
create-usb: Add a --commit option
Currently on Endless OS, the OSTree ref for the operating system is
something like os/eos/amd64/eos3, so that's what gets passed to `ostree
create-usb` when copying the OS to a USB drive (for offline updates).
However, when eos-updater checks for updates it pulls the metadata for a
candidate commit and in so doing updates that eos3 ref to point to the
partial commit being examined as a potential update rather than the
deployed commit. This causes `ostree create-usb` to fail with an error
like "No such metadata object 7fb045cb2d1f1f3a81bfc157c6128ff443eb56350315b9536bdb56aee0659863.dirtree".
OSTree creates deployment refs that look like "ostree/1/1/0" to maintain
a pointer to the deployed commit, but create-usb can't use these because
it shows up in the summary as just a ref, not a collection-ref.
So this commit adds a --commit option to the create-usb command, so we
can use the appropriate ref but copy the deployed commit rather than a
(potentially partial) update commit.
Colin Walters [Mon, 24 Sep 2018 17:37:29 +0000 (13:37 -0400)]
lib/commit: Don't copy xattrs for metadata objects
Copying the xattrs on metadata objects is wrong in general, we
don't "own" them. Notably this would fail in the situation of
doing a pull from e.g. a `bare-user` source to a destination
that was on a different mount point (so we couldn't hardlink),
and the source had e.g. a `security.selinux` attribute.
Matthew Leeds [Fri, 21 Sep 2018 22:35:50 +0000 (15:35 -0700)]
create-usb: Always use archive mode
Change the create-usb command so that it always creates the destination
repository using the "archive" mode, rather than using archive mode when
xattrs aren't supported and bare-user otherwise. This has a few
advantages:
1. The archive mode works with FAT filesystems, which is what most
USB drives are, and which doesn't support xattrs.
2. At least in some quick testing I did, archive mode is about
twice as performant as bare-user mode, in terms of how long it takes for
the create-usb command to complete.
3. This ensures that a tool can safely change the permissions on
".ostree/repo" and subdirectories after create-usb completes, which is
important for Endless since otherwise you can't use `ostree create-usb`
as root and then `flatpak create-usb` as a non-root user on the same USB
drive (or in other words copy OS updates and apps to the same USB).
Colin Walters [Fri, 21 Sep 2018 13:47:36 +0000 (09:47 -0400)]
deploy: Fix removing /var/.updated with separate /var mount
There's some subtlety to this, we don't handle all cases.
But the 99% cases are using `--sysroot deploy` to create an
initial deployment, and then doing upgrades from inside
a booted deployment.
It was only the latter case that didn't work with a separate `/var`.
Fixing all of them would probably require libostree to learn
how to e.g. look at `/etc/fstab` (or worse, systemd mount units?)
and handle the mounting. I don't think we want to do anything
like that right now, since there are no active drivers for the
use case.
lib/repo: Separate min-free-space-* calculation from transaction codepath
Earlier, the actual reserved space (in blocks) were calculated inside the
transaction codepath ostree_repo_prepare_transaction(). However, while
reworking on ostree_repo_get_min_free_space_bytes() API, it was realized that
this calculation can be done independently from the transaction's codepaths, hence
enabling the usage for ostree_repo_get_min_free_space_bytes() API irrespective
of whether there is an ongoing transaction or not.
Matthew Leeds [Wed, 19 Sep 2018 21:59:47 +0000 (14:59 -0700)]
lib/repo: Define a metadata key, ostree.deploy-collection-id
This commit defines a metadata key that tells clients to update their
remote config to add a collection ID. This functionality is currently
implemented in Flatpak for the key "xa.collection-id", but there are two
good reasons for moving the key to OSTree:
1) Servers such as Flathub shouldn't set xa.collection-id in their
metadata now or in the medium term future, because many users are still
using old versions of Flatpak and OSTree[1] which would hit various
bugs[2][3][4] on the P2P code paths that are enabled by collection IDs.
Defining a new key means that only clients running recent
(as-yet-unreleased) versions of Flatpak and OSTree will pay attention to
it and deploy the collection ID, leaving the users on old versions
unaffected.
2) OSTree is as "invested" in collection IDs as Flatpak, so there's no
reason the key should be defined in Flatpak rather than here. According
to Philip Withnall, the reason the key was put in Flatpak originally was
that at the time there was uncertainty about tying OSTree to collection
IDs.
Robert McQueen [Tue, 18 Sep 2018 14:46:24 +0000 (15:46 +0100)]
OstreeMutableTree: add _remove method
There is no API method to remove a file or subdirectory from a MutableTree
besides directly manipulating the GHashTable returned by _get_files or
_get_subdirs. This isn't possible from an introspection binding that transforms
the returned GHashTable, and may also leave the tree checksum in an invalid
state. Introduce a new method so that removing files or subdirectories is
safe, and possible from bindings.
Jonathan Lebon [Thu, 20 Sep 2018 15:11:45 +0000 (11:11 -0400)]
ci: Disable f28-rpmostree for now
It started failing with:
```
ERROR: tests/check/test-ucontainer.sh - too few tests run (expected 2, got 0)
tap-driver.sh: internal error getting exit status
tap-driver.sh: fatal: I/O or internal error
make[4]: *** [Makefile:4353: tests/check/test-ucontainer.sh.log] Error 1
make[4]: *** Waiting for unfinished jobs....
```
And the artifacts are not being saved for some reason:
```
+ cleanup
+ mv test-suite.log /var/tmp/checkout
mv: cannot stat 'test-suite.log': No such file or directory
+ true
+ mv vmcheck /var/tmp/checkout
mv: cannot stat 'vmcheck': No such file or directory
+ true
```
Let's just disable this for now so that some other pending patches can
go in while we investigate.
Matthew Leeds [Wed, 5 Sep 2018 04:49:23 +0000 (21:49 -0700)]
avahi: Be robust to missing refs in peer summaries
In the OstreeRepoFinderAvahi implementation,
ostree_avahi_service_build_repo_finder_result() is where the DNS-SD
records are processed and turned into OstreeRepoFinderResult objects.
Each result object is supposed to have a hash table mapping refs to
checksums, so this is accomplished by first adding a placeholder (a ref
mapping to a NULL checksum) for each ref matched by the bloom filter,
and later filling in the checksums using the remote's summary file,
which happens in get_checksums(). The problem is that there's no
guarantee all the checksums will be resolved (non-NULL), so the
ostree_repo_finder_result_new() call then hits an assertion failure in
is_valid_collection_ref_map() leading to a crash (in the case that one
or more refs had NULL checksums).
There are at least two situations where the ref checksum might not be
found in the peer remote's summary file:
1) The bloom filter match was a false positive. This is going to happen
sometimes by design.
2) The peer remote's summary is out of sync with its DNS-SD records.
This shouldn't normally happen but it's still good to be robust to the
possibility; in Endless OS nothing guarantees the atomicity of updating
the summary and DNS-SD records.
This commit changes libostree to be robust to the possibility of refs
missing from the peer remote's summary, by removing any that still have
a NULL checksum associated with them after the summary has been fetched
and processed.
The other OstreeRepoFinder implementations don't have this issue because
they use summary files directly and therefore always have access to the
checksum.
Umang Jain [Fri, 31 Aug 2018 14:14:22 +0000 (19:44 +0530)]
lib/repo: Ensure min-free-space* config value doesn't overflow
when converted to bytes
In a subsequent commit, we add a public API to read the value of
min-free-space-* value in bytes. The value for free space check
is enforced in terms of block size instead of bytes. Therefore,
for consistency we check while preparing the transaction that the
value doesn't overflow when converted to bytes.
Felix Krull [Wed, 29 Aug 2018 18:23:03 +0000 (20:23 +0200)]
lib/grub2: Support Debian-style grub.cfg path
Debian and Debian-derived systems have their GRUB configuration file in
/boot/grub/grub.cfg, rather than /boot/grub2/grub.cfg. Detecting this
file is necessary to correctly generate GRUB boot configuration on
Debian systems.
Robert Fairley [Mon, 20 Aug 2018 19:40:32 +0000 (15:40 -0400)]
docs: Add Contributing Tutorial to Mkdocs pages
This adds the Contributing Tutorial (contributing-tutorial.md) to
the pages setting of Mkdocs, so that the tutorial will render
in the readthedocs.io documentation.
Robert Fairley [Thu, 16 Aug 2018 20:46:53 +0000 (16:46 -0400)]
docs: Add contributing tutorial
This adds a tutorial for developing/contributing to OSTree. The following information is included:
- Installing build dependencies for OSTree
- Building OSTree, and running in a container and VM
- Adding a basic command to OSTree, and testing the command
- Suggested git workflows for working with the upstream repo
This is helpful to give new contributors a more detailed introduction to developing OSTree, and an idea of what the workflow looks like.
Sinny Kumari [Fri, 17 Aug 2018 15:55:56 +0000 (21:25 +0530)]
src/ostree: Add --group option to ostree config
Fetching value from a repo config using 'ostree config
get SECTIONNAME.KEYNAME' didn't work in some cases like
when having dots in Group Name entry.
As per Desktop entry file specification, Group Name
may contain all ASCII characters except for [ and ]
and control characters.
Link - https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-1.1.html
Having --group option will help user to clearly specify
Group Name and get desired result.
It also adds test for ostree config get|set and bash
completion for --group option
Philip Withnall [Mon, 21 May 2018 14:11:36 +0000 (15:11 +0100)]
lib/repo-pull: Prefer object pull over from-scratch delta if ref exists
If a ref already exists, we are likely only a few commits behind the
current head of the ref, so it is probably better for bandwidth
consumption to pull the individual objects rather than the from-scratch
delta.
Signed-off-by: Philip Withnall <withnall@endlessm.com> Closes: #1709
Approved by: cgwalters
Will Thompson [Fri, 17 Aug 2018 13:40:01 +0000 (14:40 +0100)]
repo: remove outdated note from write_config() docs
Since 9dc6ddce0848cf04b76fc1c673c8fc1a0f6f425a it has not been true that
'new_config' was simply ref'd: it's serialized, and then re-parsed into
a new GKeyFile.
0 is not a valid value (the units are required), and 1MB is taken to
mean 1048576 bytes, not 1000000 bytes (unlike other human-readable sizes
in libostree, which are formatted by g_format_size(), and hence use
power-of-10 units). It's probably a bit late to change this latter
point, but the documentation should mention it.
Define 'MB' etc. more precisely; include the byte counts in the
examples; and improve the formatting of the min-free-space-* paragraphs.
Signed-off-by: Will Thompson <wjt@endlessm.com> Closes: #1706
Approved by: mwleeds
Jonathan Lebon [Sat, 18 Aug 2018 01:50:14 +0000 (21:50 -0400)]
ci: Bump rpm-ostree tag we build for tests
The latest rpm-ostree release no longer requires `python3-devel` so `dnf
builddep` here is no longer pulling it in, subsequently causing issues
when building an older rpm-ostree release. Let's just bump the release
tag so we don't have to also start keeping track of older dependencies.
Dan Nicholson [Fri, 10 Aug 2018 16:53:50 +0000 (11:53 -0500)]
repo: Add OSTREE_REPO_TEST_ERROR=invalid-cache env var
Add an invalid-cache test error flag to ensure that the code that checks
for and recovers from a corrupted summary cache is hit. This helps make
sure that the recovery path is actually used without resorting to
G_MESSAGES_DEBUG.
Dan Nicholson [Thu, 2 Aug 2018 20:15:28 +0000 (15:15 -0500)]
lib/pull: Fetch summary if cached version doesn't match signature
If for some reason the cached summary doesn't match the cached signature
then fetch the remote summary and verify again. Since commit c4c2b5eb
this is unlikely to happen since the summary will only be cached if it
matches the signature. However, if the summary cache has been corrupted
for any other reason then it's best to be safe and fetch the remote
summary again.
This is essentially the corollary to c4c2b5eb. Where that commit helps
you from getting into the corrupted summary cache in the first place,
this helps you get out of it. Without this the client can get wedged
until a prune or the remote server republishes the summary.
Dan Nicholson [Thu, 2 Aug 2018 18:58:56 +0000 (13:58 -0500)]
tests: Add tests for remote summary update races
There have been subtle bugs in the past when a client pulls while the
remote server is updating the summary. The client may get the old
summary and new signature or vice versa. Add tests to simulate this
behavior to make sure there aren't regressions in the future.
Now that we have `auto-update-summary`, there is no point in having
`commit-update-summary`. The latter also only had an effect through
the `commit` CLI command, whereas the former is embedded directly in
libostree.
There is one corner case that slips through: `commit` would update the
summary file even if orphan commits were created, which we no longer do
here. I can't imagine anyone relying on this, so it seems safe to drop.
Jonathan Lebon [Mon, 30 Jul 2018 14:46:58 +0000 (10:46 -0400)]
lib/config: Rename change-update-summary to auto-...
Mildly bikeshed, though I find the name `auto-update-summary` to be
easier to grok than `change-update-summary`. I think it's because it can
be read as "verb-verb-noun" rather than "noun-verb-noun".
Matthew Leeds [Tue, 31 Jul 2018 03:05:56 +0000 (20:05 -0700)]
lib/repo-pull: Use correct keyring for dynamic remotes
Normally, a configured remote will only serve refs with one associated
collection ID, but temporary remotes such as USB drives or LAN peers can
serve refs from multiple collection IDs which may use different GPG
keyrings. So the OstreeRepoFinderMount and OstreeRepoFinderAvahi classes
create dynamic OstreeRemote objects for each (uri, keyring) pair. So if
for example the USB mounted at /mnt/usb serves content from the
configured remotes "eos-apps" and "eos-sdk", the OstreeRepoFinderResult
array returned by ostree_repo_find_remotes_async() will have one result
with a remote called something like
file_mnt_usb_eos-apps.trustedkeys.gpg and the list of refs on the USB
that came from eos-apps, and another result with a remote
file_mnt_usb_eos-sdk.trustedkeys.gpg and the list of refs from eos-sdk.
Unfortunately while OstreeRepoFinderMount and OstreeRepoFinderAvahi
correctly only include refs in a result if the ref uses the associated
keyring, the find_remotes_cb() function used to clean up the set of
results looks at the remote summary file and includes every ref that's
in the intersection with the requested refs, regardless of whether it
uses a different remote's keyring. This leads to an error when you try
to pull from a USB containing refs from different collection IDs: the
pull using the wrong collection ID will error out with "Refspec not
found" and the result with the correct keyring will then be ignored "as
it has no relevant refs or they have already been pulled." So the pull
ultimately fails.
This commit fixes the issue by filtering refs coming from a dynamic
remote, so that only ones with the collection ID associated with the
keyring remote are examined. This only needs to be done for dynamic
remotes because you should be able to pull any ref from a configured
remote using its keyring. It's also only done when looking at the
collection map in the summary file, because LAN/USB remotes won't have a
"main" collection ID set (OSTREE_SUMMARY_COLLECTION_ID).
Colin Walters [Tue, 31 Jul 2018 21:04:26 +0000 (17:04 -0400)]
ostree-remount.service: RemainAfterExit=yes
This is standard practice for units like this; e.g. it's what
`systemd-remount-fs.service` does. I think it may be part of
or the whole cause for
https://github.com/projectatomic/rpm-ostree/issues/1471
I haven't reproduced the problem exactly but it seems to me that
if the unit starts and is GC'd, then when systemd goes to execute
a later unit it might end up restarting it.
A noticeable side effect of this is that `systemctl status ostree-remount`
exits with code `0` as expected.
Matthew Leeds [Fri, 13 Jul 2018 22:53:21 +0000 (15:53 -0700)]
config: Add a core/change-update-summary option
This commits adds and implements a boolean repo config option called
"change-update-summary" which updates the summary file every time a ref
changes (additions, updates, and deletions).
The main impetus for this feature is that the `ostree create-usb` and
`flatpak create-usb` commands depend on the repo summary being up to
date. On the command line you can work around this by asking the user to
run `ostree summary --update` but in the case of GNOME Software calling
out to `flatpak create-usb` this wouldn't work because it's running as a
user and the repo is owned by root. That strategy also means flatpak
can't update the repo metadata refs for fear of invalidating the
summary.
Another use case for this relates to LAN updates. Specifically, the
component of eos-updater that generates DNS-SD records advertising ostree
refs depends on the repo summary being up to date.
Since ostree_repo_regenerate_summary() now takes an exclusive lock, this
should be safe to enable. However it's not enabled by default because of
the performance cost, and because it's more useful on clients than
servers (which likely have another mechanism for updating the summary).
Matthew Leeds [Mon, 16 Jul 2018 21:54:57 +0000 (14:54 -0700)]
lib/repo: Take exclusive lock while generating summary
This ensures that commits aren't deleted and refs aren't added, removed,
or updated while the summary is being generated. This is in preparation
for adding a repo config option that will automatically regenerate the
summary on every ref change.
Jonathan Lebon [Wed, 25 Jul 2018 21:51:01 +0000 (17:51 -0400)]
lib: Fix some logic/error-checking code
Using `MAX(0, $x)` here is useless since we're comparing against an
unsigned integer. Just unpack this and only subtract if it's safe to do
so.
Also, explicitly check for `fd >= 0` rather than just `!= -1` to be sure
it's a valid fd. And finally, explicitly check the return value of
`g_input_stream_read_all` as is done everywhere else in the tree and
make it clear that we're purposely ignoring the return value of `_flush`
here, but not in other places.
Jonathan Lebon [Wed, 25 Jul 2018 21:49:12 +0000 (17:49 -0400)]
lib: Fix some minor memory leaks
I initially was going to add a `G_DEFINE_AUTOPTR_CLEANUP_FUNC` for
`FetchStaticDeltaData`, but it honestly didn't seem worth mucking around
ownership everywhere and potentially getting it wrong.
Now that it's possible to have both min-free-space-size and
min-free-space-percent set in a repo config, update the docs to make
the behavior clear in that case.
Matthew Leeds [Tue, 24 Jul 2018 21:52:24 +0000 (14:52 -0700)]
lib/repo-commit: Fix min-free-space error message
Since min_free_space_size_mb is considered before min_free_space_percent
in min_free_space_calculate_reserved_blocks(), it has to be considered
first when generating the error message in order for it to be accurate.
Colin Walters [Fri, 20 Jul 2018 18:13:02 +0000 (14:13 -0400)]
ci: Mark insttests as not required
The reliablity has just not been what we need, and they haven't
yet caught any real bugs. Until I can carve off some time to
truly make this reliable let's just mark it as not required.
I'd like to gather more statistics on the failure scenarios.
lib/repo: Allow min-free-space-size and -percent to co-exist
Previously, we would error out if both of the options were mentioned
in the config file (even if one of them is disabled with 0). There
were few suggestions that this behavior was not quite right.
Therefore, instead of throwing error and exiting, it's preferred to
warn the user. Hence, the solution that worked out is:
* Allow both options to exist simulateneously
* Check each config's value and decide:
* If both are present and are non-zero, warn the user. Also, prefer
to use min-free-space-size over the another.
* If both are absent, then use -percent=3% as fallback
* Every other case is valid hence, no warning
https://phabricator.endlessm.com/T13698
(cherry picked from commit be68991cf80f0aa1da7d36ab6e1d2c4d6c7cd3fb) Signed-off-by: Robert McQueen <rob@endlessm.com> Closes: #1685
Approved by: cgwalters
Some downstreams — namely, the Yocto Project — ship gpg-error with a
pkg-config file, and modify gpg-error-config to error out when you try
using it instead of pkg-config.
We can check for gpg-error via pkg-config, and if it's not available,
fall back to gpg-error-config.
Colin Walters [Mon, 16 Jul 2018 18:43:17 +0000 (14:43 -0400)]
build-sys: Link with gpg-error directly
We use the API, and not linking breaks the build with e.g.
`-fuse-ld=gold` in a Fedora 28 buildroot as gold doesn't do the
"search indirect dependencies" thing.
Needed to fix `rpm-ostree kargs` test suite with default staging; skipping
a test here for now as eventually what we'll do is turn on the rpm-ostree
suite fully here.
William Manley [Fri, 22 Jun 2018 14:28:49 +0000 (15:28 +0100)]
ostree repo commit: Speed up composing trees with `--tree=ref`
Running `ostree commit --tree=ref=a --tree=dir=b` involves reading the
whole of a into an `OstreeMutableTree` before composing `b` on top. This
is inefficient if a is a complete rootfs and b is just touching one file.
We process O(size of a + size of b) directories rather than
O(number of touched dirs).
This commit makes `ostree commit` more efficient at composing multiple
directories together. With `ostree_mutable_tree_fill_empty_from_dirtree`
we create a lazy `OstreeMutableTree` which only reads the underlying
information from disk when needed. We don't need to read all the
subdirectories just to get the checksum of a tree already checked into the
repo.
This provides great speedups when composing a rootfs out of multiple other
rootfs as we do in our build system. We compose multiple containers
together with:
time ostree --repo=... commit --orphan --tree=ref=big-rootfs --tree=dir=modified_etc
Where modified_etc contained a modified sudoers file under /etc. I used
`strace` to count syscalls and I seperatly took timing measurements. To
test with a cold cache I ran
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
I'm not sure if this will have some negative interaction with the
`_ostree_repo_commit_modifier_apply` which is short-circuited here. I
think it was disabled for `--tree=ref=` anyway, but I'm not certain. All
the tests pass anyway.
I originally implemented this in terms of the `OstreeRepoFile` APIs, but
it was *way* less efficient, opening and reading many files unnecessarily.
Alex Kiernan [Sat, 7 Jul 2018 21:35:35 +0000 (21:35 +0000)]
switchroot: Move late /run/ostree-booted creation to ostree-system-generator
When ostree-prepare-root is pid 1, ostree-prepare-boot defers creation of
/run/ostree-booted, which happens in ostree-remount, but that's too late
if we need ostree-system-generator to bind /var. Add the creation of the
/run/ostree-booted marker to ostree-system-generator based on the
existence of the ostree= kernel command line argument (which matches the
condition that ostree-remount uses).
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Closes: #1675
Approved by: cgwalters
Colin Walters [Tue, 3 Jul 2018 21:28:48 +0000 (17:28 -0400)]
deploy: Retain staged by default
For `rpm-ostree ex livefs` we have a use case of pushing a rollback
deployment. There's no reason this should require deleting the staged
deployment (and doing so actually breaks livefs which tries to access
it as a data source).
I was initially very conservative here, but I think it ends up
being fairly easy to retain the staged deployment. We need to handle
two cases:
First, when the staged is *intentionally* deleted; here, we just need
to unlink the `/run` file, and then everything will be sync'd up after
reloading.
Second, (as in the livefs case) where we're retaining it,
e.g. adding a deployment to the end. What I realized here is that
we can have the code keep `new_deployments` as view without staged,
and then when we do the final reload we'll end up re-reading it from
disk anyways.
Robert Fairley [Thu, 5 Jul 2018 18:59:24 +0000 (14:59 -0400)]
tests: Run network retries test for many retries directly
This runs a test involving many retries for the --network-retries
option directly rather than inside a conditional statement, so that
the command does not silently fail and allow the test to continue
running.
Robert Fairley [Thu, 5 Jul 2018 16:35:30 +0000 (12:35 -0400)]
ostree/pull: Add network-retries command line option
This exposes a way to specify from the command line the number
of times to retry each download after a network error. If a negative
value is given, then the default number of retries (5) is used. If 0
is given, then errors are returned without retrying.
Robert Fairley [Thu, 5 Jul 2018 16:30:05 +0000 (12:30 -0400)]
tests: Move assert_fail function to tests/libtest.sh
This moves the assert_fail function definition which was defined
and called in tests/test-remote-headers.sh. Done in preparation
for use of the assert_fail function in other test files.
Alex Kiernan [Wed, 4 Jul 2018 19:19:41 +0000 (19:19 +0000)]
build: Use ostree_prepare_root_CPPFLAGS for ostree-prepare-root
Swap from AM_CPPFLAGS to ostree_prepare_root_CPPFLAGS when compiling
ostree-prepare-root statically. This fixes a problem when you have
systemd and libmount, but only ostree_prepare_root_CPPFLAGS includes
-DHAVE_SYSTEMD_AND_LIBMOUNT=1.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Closes: #1670
Approved by: jlebon
Jonathan Lebon [Wed, 4 Jul 2018 15:59:18 +0000 (11:59 -0400)]
tests/installed: Add NOTE when re-using RPMs
One gotcha here is that we don't invalidate the RPMs if we're not
sitting on the same commit anymore. Shouldn't be too hard to fix, though
let's at least make a note of it for now.
build: add ostree-soup-* to build process when configured with avahi
Avoid getting these link errors:
./.libs/libostree-1.so: undefined reference to `soup_uri_set_path'
./.libs/libostree-1.so: undefined reference to `soup_uri_new'
./.libs/libostree-1.so: undefined reference to `soup_uri_free'
./.libs/libostree-1.so: undefined reference to `soup_uri_set_scheme'
./.libs/libostree-1.so: undefined reference to `soup_uri_to_string'
./.libs/libostree-1.so: undefined reference to `soup_uri_set_host'
./.libs/libostree-1.so: undefined reference to `soup_uri_set_port'
collect2: error: ld returned 1 exit status
Alex Kiernan [Sat, 30 Jun 2018 17:03:51 +0000 (17:03 +0000)]
boot: Use emergency.target, not emergency.service
Follow systemd units in using emergency.target, not emergency.service
(which is the sole unit, by default, in emergency.target) so we can
easily reconfigure the units which are actived when entering
emergency mode.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Closes: #1665
Approved by: cgwalters
William Manley [Mon, 25 Jun 2018 20:53:23 +0000 (21:53 +0100)]
OstreeMutableTree: Refactor: Add `parent` pointer
This implements a TODO item from
`ostree_mutable_tree_get_contents_checksum`. We now no-longer invalidate
the dirtree contents checksum at `get_contents_checksum` time - we
invalidate it when the mtree is modified. This is implemented by keeping
a pointer to the parent directory in each `OstreeMutableTree`. This gives
us stronger invariants on `contents_checksum`.
For even stronger guarantees about invariants we could make
`ostree_repo_write_mtree` or similar a member of `OstreeMutableTree` and
remove `ostree_mutable_tree_set_metadata_checksum`.
I think I've fixed a bug here too. We now invalidate parent's contents
checksum when our metadata checksum changes, whereas we didn't before.
Colin Walters [Fri, 29 Jun 2018 17:53:54 +0000 (13:53 -0400)]
ci/flatpak: Fix to use built ostree version
Noticed as part of a random failure in this PR:
https://github.com/ostreedev/ostree/pull/1655
that we weren't actually testing the version of ostree
we built in git. Probably we should be generating RPMs but...later.
Philip Withnall [Tue, 26 Jun 2018 13:39:16 +0000 (14:39 +0100)]
lib/repo-commit: Abort a transaction if preparing it fails
If ostree_repo_prepare_transaction() fails, we should reset the
repository’s state so that the failed call was essentially idempotent.
Do that by calling ostree_repo_abort_transaction() on the failure path.
Typically, the way for preparing a transaction to fail is for its
GCancellable to be triggered, rather than because any of the operations
involved in preparing a transaction are particularly failure prone.
Signed-off-by: Philip Withnall <withnall@endlessm.com> Closes: #1647
Approved by: cgwalters
Umang Jain [Tue, 26 Jun 2018 21:10:15 +0000 (02:40 +0530)]
lib/repo: Cleanup current boot's staging dir min-free-space-* checks are hit
min-free-space-* act as a gating condition whether to we want hold onto caches in
repo/tmp. If it is found that the free-disk space is going below this threshold,
we flag it as an error and cleanup current boot's staging directory.
Umang Jain [Tue, 26 Jun 2018 21:11:43 +0000 (02:41 +0530)]
lib/repo: cleanup_tmpdir should be executed after releasing lock file
Here's a subtle bug in abort_transaction():
One of the policies of cleaning up is to skip the current boot's staging
directory. The responsible function for this is cleanup_tmpdir() which tries
to lock each of the tmpdir before deleting it. When it comes to the current
boot's staging dir, it tries to lock the directory(again!) but fails as there
is already a lockfile present. Just because the current boot's staging dir was
meant to be skipped, the bug never surfaced up and wasn't catastrohpic.
if (!_ostree_repo_try_lock_tmpdir (dfd, path, &lockfile, &did_lock, error))
return FALSE;
if (!did_lock)
return TRUE; /* Note early return */
...
if (g_str_has_prefix (path, self->stagedir_prefix))
return TRUE; /* Note early return */
The actual check for skipping staging dir for current boot was never reached
because the function returned at did_lock failure.
Therefore, execute cleanup_tmpdir() after releasing the lockfile in
abort_transaction() so that cleanup_tmpdir gets a chance to lock current boot's
staging directory and succeed.
deploy: Change BootLoaderSpec filenames so they can be used for sorting
Currently the BLS snippets are named ostree-$ID-$VARIANT_ID-$index.conf,
but the BLS config files are actually sorted by using the version field
which is the inverse of the index.
In most places, _ostree_sysroot_read_boot_loader_configs() is used to
get the BLS files and this function already returns them sorted by the
version field. The only place where the index trailing number is used is
in the ostree-grub-generator script that lists the BLS files to populate
the grub config file.
But for some bootloaders the BLS filename is the criteria for sorting by
taking the filename as a string version. So on these bootloaders the BLS
entries will be listed in the reverse order.
To avoid that, change the BLS snippets filename to have the version field
instead of the index and also to have the version before deployment name.
Make the filenames to be of the form ostree-$version-$ID-$VARIANT_ID.conf
so the version is before the deployment name.
Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Closes: #1654
Approved by: cgwalters
Simon McVittie [Sun, 24 Jun 2018 11:56:49 +0000 (12:56 +0100)]
OstreeRepoFinderConfig: Fix guint/gsize confusion
If a function has a guint "out argument", passing a pointer to a gsize
is not, in general, valid. On an ILP64 platform there is no problem
since guint and gsize are identical, but on an LP64 platform it will
overwrite only the first word of the gsize, leaving the second word
unaffected. On little-endian machines, if the second word is
zero-initialized (as it is here), the result is numerically equal to
the guint, but on big-endian machines the result is around 4 billion
times what it should be, resulting in
ostree_repo_finder_config_resolve_async() reading past the end of
the array and causing undefined behaviour.
In practice this caused assertion failures (and consequently test
failures) on Debian's s390x (z/Architecture), ppc64 (64-bit PowerPC)
and sparc64 (64-bit SPARC) ports.
ostree-grub-generator: sort BLS files by version instead of alphabetically
The ostree-grub-generator populates the grub.cfg menu entries using the
BLS config files. But it uses the ls command that by default sorts the
entries alphabetically, so the order won't be correct if there are more
than 10 deployments, i.e:
$ ls -1 /boot/loader/entries/
ostree-fedora-workstation-0.conf
ostree-fedora-workstation-10.conf
ostree-fedora-workstation-1.conf
...
So instead the -v option should be used to make ls use version sorting:
$ ls -1 -v /boot/loader/entries/
ostree-fedora-workstation-0.conf
ostree-fedora-workstation-1.conf
...
ostree-fedora-workstation-10.conf
Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Closes: #1653
Approved by: cgwalters
William Manley [Fri, 22 Jun 2018 10:37:37 +0000 (11:37 +0100)]
OstreeMutableTree: Document each private member of `OstreeMutableTree`
A prelude to my understanding. Unfortunately `OstreeMutableTree` provides
little encapsulation, as each member has setters† so it's difficult to come
up with a list of invariants.
† `files` and `subdirs` only have getters, but the getters return mutable
references to the internals, so we still can't reason about invariants.
Simon McVittie [Fri, 22 Jun 2018 00:10:55 +0000 (01:10 +0100)]
avahi: Fail immediately if we can't talk to D-Bus or Avahi
We special-case AVAHI_ERR_NO_DAEMON to not cause warnings, but if
we pass AVAHI_CLIENT_NO_FAIL to avahi_client_new, we never actually
see AVAHI_ERR_NO_DAEMON. Instead, we will get AVAHI_ERR_BAD_STATE
when we try to use the client.
projects / ostree.git / log